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Abstract — In this paper, we propose an alternative for routing 
based packet forwarding, which uses network coding to increase 
transmission efficiency, in terms of both compression and er- 
ror resilience. This non-adaptive encoding is called quantized 
network coding, which involves random linear mapping in the 
real field, followed by quantization to cope with the finite 
capacity of the links. At the gateway node, which collects received 
quantized network coder packets, minimum mean squared error 
decoding is performed, by using belief propagation in the factor 
graph representation. Our simulation results show a significant 
improvement, in terms of the number of required packets to 
recover the messages, which can be interpreted as an embedded 
distributed source coding for correlated messages. 

Index Terms — Network coding, Bayesian compressed sensing, 
belief propagation, minimum mean squared error estimation. 

I. Introduction 

Data gathering in sensor networks has drawn attention to 
network coding 1 1 ] as an alternative for routing based packet 
forwarding |2 ] because of its flexibility, and robustness to the 
network changes and link failures. In the case of correlated 
messages, performing network coding on top of distributed 
source coding (3) is shown to be optimal, in terms of the 
achievable information rates |4|. However, appropriate encod- 
ing rates have to be known at the encoders, which requires 
transmission of an overhead and affects the flexibility and 
distributed nature of sensor networks. 

Recently, the possibility of adopting non-adaptive joint 
source network coding has been studied by using the concepts 
of compressed sensing [5 ] and sparse recovery (6|-|TT|. As 
a first major work to formulate and investigate theoretical 
feasibility of compressed sensing based network coding, we 
proposed to use Quantized Network Coding (QNC) |TQ| , fT2| , 
which involves random linear network coding in the real field 
and quantization. In fT0| , our decoding scheme was based 
on ti -minimization (using linear programming 1 13 ]), which is 
shown to be optimal for recovery of exactly sparse messages, 
from noiseless measurements. In this paper, we study optimal 
Minimum Mean Squared Error (MMSE) decoding and fea- 
sibility of its implementation in practical cases. Specifically, 
we propose to use a near optimal MMSE decoding based on 
Belief Propagation (BP). 

Quantized network coding using low density coefficients is 
described and formulated in section [n| We discuss optimal 
MMSE decoding for our QNC scenario with known priori in 
section [TTTJ In section [IVJ we describe our BP based MMSE 
decoding, followed by our simulation results in sectio n |V| 
Finally, our concluding remarks are presented in section |VI| 



II. Quantized Network Coding 

Consider a network (graph), Q = (V, £), with the set of 
nodes V = {1, . . . , n}, and the set of directed edges (links) 
£ = Each edge, e, can maintain a lossless 

transmission from tail(e) to head(e), at a maximum rate of 
C e bits per use. The input content of edge e (which is the 
same as its output content), at time index t, is represented 
by y e (t), and is from a finite alphabet of size 2 LCe , where 
L is the block length, transmitted in each time slot, between 
t — 1 and t. For each node, v, we define sets of incoming 
edges, In(v) = {e : head(e) — v, e G £}, and outgoing edges, 
Out(v) = {e : tail(e) = v, e G £}. Moreover, each node v has 
a random information source, X v , where there is a transform 
matrix, </> nX n> such that X_ = [X v : v G V] and S_ = <j) T X_ is 
/c-sparse. As in the rest of this paper, where we represent the 
realizations of random variables with lower case letters, the 
outcome realization of X_ is represented by x. In this paper, 
we study the (single session) data gathering, where all the 
messages, X v 's, are to be transmitted to a single node, called 
decoder (or gateway), vo eV. 

We defined QNC at each node, v G V, as follows |T0|: 



Ye(t) = Qe[ 



e' £ln(v) 



P ei e>(t)-Y e (t-l)+a eiV (t)<X % 



(1) 



where Q e [.] is the quantizer (designed based on the value 
of C e and L, and the distribution of incoming contents and 
messages), associated with the outgoing edge e G Out(v). 
The corresponding network coding coefficients, /3 eje /(t) and 
a &iV (i) are selected from real numbers: /3 eje r(i),a ejV (i) G R, 
and satisfy the normalizing condition of (3) in [10]. Initial rest 
condition is also assumed to be satisfied in our QNC scenario: 
y e (l) = 0, VeGf. 

Denoting the quantization noise at edge e by N e (t), we 
have: 

Y e (t)= Pe,e>(t).Y e (t-l) + a e , v (t)-X v +N e (t). (2) 

e' Eln(v) 

This is equivalent to: 



Y(t) = F(t) ■ Y(t - 1) + A(t) ■ X + N(t), 



(3) 



where N(t) = [N e (t) : e G £ ], and: 

F(t) ' tF(i)\ > = i ' tail ^ = head ( e ') 

\ )\e\x\s\ • 1 v )fe,e' | q ^ otherwise 



Representing the marginal measurements (received packets 
to the decoder) at time t, by Z_{t), we have: 



Z(t) = [Ye(t):e€ln(v )]=B-Y(t), 



(4) 



where: 



r i , i 

\ , c 



corresponds to e, e G In(vo) 
otherwise 



We store marginal measurements, over time, and build up a 
tofa/ measurements vector, Z_ tot (t): 



z tot (t) 



1(2) 



W) 



, m = (t - l)\In(v )\. (5) 



mxl 



As a result of linearity of QNC scenario (Eq.[3]), we have |T0| : 

Z tot (t) = * tot (*) • X + iVeff ,tot (*) . ( 6 ) 

where ^ tot (t) and 7V eff tot (t) are called total measurement 
matrix and total effective noise vector, respectively. 

In (TO), a compressed sensing (i.e. ^i-min) decoding is used 
to reconstruct X_ from noisy under-determined measurements, 
{^tot(^)}' s - Being able to recover n different values, X v 's, 
from m measurements, where m is usually much less than 
n, can be interpreted as an embedded distributed compression 
of inter-node correlated X v 's. Although this is feasible with 
respect to some distortion, the proposed ^i-min decoding does 
not offer an optimal solution, especially when a prior on X_ 
is available and has more information than sparsity. In this 
paper, we address the optimal MMSE decoding in a Bayesian 
QNC scenario by studying the computational complexity of 
implementing such decoder. Motivated by the work in p4) , 
near optimal implementation of MMSE decoding based on 
belief propagation and the appropriate design of network 
coding coefficients are discussed. 

III. Minimum Mean Square Error Decoding 

The a priori model used to characterize the messages is a 
Gaussian mixture model. Specifically, we consider states, Q v 's 
(S v is the f'th element of vector S), corresponding to S v 's, 
which are independent binary random variables with: 

P(Q„ = 1) = -, WgV. (7) 

n 

Each state, Q v , determines if S v is zero or not: 

Qv = 1 -> S v ~A/"(0,a s 2 ), 

Q v = S v = 0. (8) 

Therefore, a priori of independently modelled S v 's is as 
follows: 

Ps v M = *— L=e _5i ? a? + (1 " (9) 



where S(.) is Dirac delta function, and it also implies: 



E[X 2 } = E[S 2 V ] = -a 2 , to. 



To facilitate the use of notations, we define Q e ^ v (t) to be the 
transform coefficient from X v to Y e (t) , describing the transfer 
of information of messages through the network. When the 
quantization noises, iV e (t)'s, have small variance compared 
to that of the signal, E[||X||2], the variance of Y e (t)'s can 
be approximated with the variance of noiseless propagated 
information, that is: 



(id 



Moreover, iV e (t)'s are approximately independent and their 
variance, E[N^(t)], is proportional with the variance of cor- 
responding quantizer input, E[F e 2 (t)]. Hence: 

7 n 

a 2 e (t)=E[N 2 (t)} ~ - 7e (t)a 2 s J2^U^ < 12 ) 
n 

v—l 

where j e (t) is a positive scalar, depending on the quantizer 
design. Defining 

jV(2) 



K tot (t) = 



(13) 



(t-l)|£|xl 



the effective total measurement noise, N eS tot (t), can be 
formulated according to: 



iV effitot (i)=* N ,tot(i)-iV tot (t). 



(14) 



This implies: 



EliVtotWASt.totW] = *N,tot(t)A (t)*S )tot (t). d5) 

where Aq (t) is the diagonal covariance matrix of quantization 
noises: 

A Q (t)=E[N tot (t)-N tot (t) T ]- (16) 



The MMSE estimation of X_ is calculated according to: 



i^opt 



(t) 



E 



t(t) 



(17) 
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+ OQ 



z tot (t) 



t (t) 



= <t> I 8 pJ 

J — oo 



£tat(*) = *tat(*) ) 



where: 
and, 



£tot(*) 



t (*)) 



PsU) Pz tot (t)(^tot(*) 



5 



p 2 tot (t) 



(*tot(*)) 



Pz tot (t) (*tat(*)) = Pz tot (t) (*tat(*)|£ = s) P g(s) • ds. 

Now, having a prior of 5 (Eq. [9]), the distribution of quan- 
tization noises, and the measurement equation of ([6]), one 
could calculate the posterior probability of X_ and its MMSE 
estimation, x opt (t). However, this requires a high computa- 
(10) tional complexity for the decoder, which makes it practically 



infeasible. To tackle this issue, near optimal MMSE decoding 
by using Belief Propagation (BP) is proposed fT4) . Such 
decoders are based on sum product algorithm (15), which is 
widely used in the literature of low density parity check codes. 



In section IV we describe the BP based near optimal MMSE 
decoder, used to recover messages in the considered Bayesian 
framework. 

IV. MMSE Decoding via Belief Propagation 

Belief propagation's used to calculate an approximate ver- 
sion of posterior probability, where a low density factor graph 
representation of random linear measurements is available 
|T4| . In (16), BP decoding is extended to recover from random 
linear measurements, even when the graph representation is 
dense. 

Consider the QNC measurement equation of ([6]) where 
the elements of total effective noise, {A[ eff tot (t)}^'s, are 
dependent. By eigen decomposition of their covariance matrix, 

E[iV eff)tot (i)iV e r ff)tot (t)] = U N (t) ■ A N (t) • Ul{t), (18) 



we define: 



and 



£tot(*) = Ajv 3 (*)£#(*) • Z tot (t), 



iVktotW = A^ 3 (*)£/£(*) -JVeff.totW, 

for which we have: 



(19) 
(20) 

(21) 
(22) 



In 21 & tot (t) is as follows: 

e; ot (i) = A-^)[/£(i)-* tot (i)-0, 

and {jVg ff tot W V s are uncorrected with unit variance. We 
also assume the marginal quantization noises, iV e (t)'s, can fit 
into a Gaussian distribution. As a result of this assumption, 
{iVg ff tot (t)}i's are independent zero mean Gaussian random 
variables with unit variance. 



The equivalent linear measurement equation of (21 ), which 
characterizes the QNC scenario, can be represented by a factor 
graph, as shown in Fig. [T] In this graph, each constraint 
node, v, 1 < v < n, (gray node) is connected to a subset 
of variable nodes (white nodes), i, 1 < i < m, for which 
{@tot(t)}i, v 7^ 0. After enough passings of the beliefs, 
between the nodes of the factor graph, an approximate version 
of posterior probability of S v 's may be obtained (14), (16). 

In the following, we describe BP based decoding for our 
Bayesian QNC scenario: 

1) The variable nodes have their prior information, i.e. 
Ps (■)> as an initial belief to start with. Explicitly, 
node v sends this probability density function (PDF), 
pj_^(«) = P^(-)' t0 i ts neighbour constraint nodes, 
neib(v) = {%' :l < i' < m, {& tat (t)}i, tV ^ 0}. 

1 in some cases also referred as message passing 



{ZLtWh ~ I v .-:r :,.'' }■ = E^otWk* S v 



z! tot (t) 




Fig. 1. QNC can be represented by using a factor graph from the sparse 
messages to the noisy measurements. 



2) The received beliefs at the constraint node i, as well 
as the corresponding measurement, Z_ tot (t) i = z tot (t)^ 
are used to calculate a backward belief. Specifically, 
for each v, where v G neib(i) = {v' : 1 < v' < 
n > {®tot(t)}i,yj 7^ Qj, ([2 3} leads us to the update 

PiV'O i s m e PDF of a 



equation in (24) 



In ([24]) 

zero mean Gaussian random variable with unit variance; 
★ and r represent the convolution operator, and the 
iteration index, respectively. 
3) At the variable node v, given the received backward 
beliefs from the neighbour nodes, p[_^ v (.), and a priori 
of S v , the forward beliefs are updated according to: 

p£>„) = c • p Si (s v ) ■ UvUv'M, (25) 

v' 

where c is a constant, assuring the unit integral of 
Puii(')- Given the posterior probabilities, one may 
calculate the BP based MMSE estimate of S v 's\ 



■tot 



(t)} 



SI = E T [S v \Z tot (t) = z 

r+oo 



(26) 



and the corresponding X = <f>-§ , as an approximation 

4) This procedure is repeated by going back to step 2 until 
some convergence criterion, such as: 

-l 



X -X 



(27) 



is met (where e rec controls the precision of decoding). 

V. Simulation Results 

In this section, we evaluate the performance of the pro- 
posed QNC by comparing it with the conventional routing 
based packet forwarding. Specifically, we generate random 
deployments of a network with n = 100 nodes and \S\ = 
400, 800, 1200 uniformly distributed edges, where one of 

2 In BP update stage, the incoming beliefs (messages) to a node are assumed 
to be independent. 



^ {ei„ t (t)} ! /^ + {e; ot (t)} 1 ,„s„ + {iv: ff , tot w} i (23) 



v'y^v, v' Eneib(i) 



G neifc(i) (24) 




Fig. 2. SNR versus delivery delay of QNC and Packet Forwarding for different sparsity factors and edge densities, using dense measurement matrices. 



the nodes is randomly picked to be the gateway (decoder) 
node. For each deployment, we also randomly generate 5's, 
with a mixture Gaussian distribution, as described in ([9]). 
The messages are derived from different sparsity factors of 



k = 0.05,0.15, and cr 2 s = 5 [j Furthermore, the sparsifying 
matrix, </>, is randomly generated and orthonormal. 

3 Note that the value of does not affect the results, since it will scale 
the variance of quantization noises too. 



For each deployment, we run QNC with different block 
lengths, L, and decoded the received packets at the decoder 
node to obtain x(t). The network coding coefficients, a eiV (t)'s 
and /3 e>e /(t)'s, used in QNC scenario, are generated such that 
the resulting Sfr t ot(t) is a dense Gaussian matrix. Specifically, 
a e>t; (2)'s are derived from independent zero mean Gaussian 
distributions, and the rest of a e}V (t)'s, t > 2, are set to zero. 
Moreover, /3 eje '(£)'s are chosen to be locally orthonormal, as 
described explicitly in theorem 3.1 in [10]. BP based MMSE 
decoding as well as ^i-min decoding are used to reconstruct 
the messages. The BP based decoder is as described in 
section IIVJ and is implemented by using the implementation 



in p6| |. ^i-min decoding is described in flQ| , theorem 4.1, and 
uses the open source optimization toolbox in (XT). 

For each deployment, we also simulate packet forwarding 
to transmit messages to the decoder node. The route used to 
forward the packets is optimized (in terms of delivery delay) 
and calculated using the Dijkstra algorithm (18). Continuous 
value messages are also quantized at the source nodes, by 
using a uniform quantizer with 2 LCe levels. 

For each SNR (quality), the best choice of block length, L, 
is found for both QNC and packet forwarding scenarios and 
used to minimize the corresponding delivery delay. We present 
the results, by averaging them over different realizations of 
network deployments and messages. In Fig. [2] the resulting 
average SNR is depicted versus the average delivery delay, 
obtained for different sparsity factors and density of edges, 
using dense measurement matrices. 

As shown in Figs. 2(a)|2(b)|2(c) the performance of using 
QNC is better than that of using routing based packet for- 
warding. Specifically, the adopted ^i-min decoder, proposed 
in (TO), already outperforms packet forwarding for all SNR 
values. Using BP based MMSE decoding helps us improve the 
performance for some (especially low) SNR values, compared 
to ^i-min decoding. Moreover, as it is expected, when the 
sparsity factor, - , of messages increases (meaning higher cor- 
relation between X v 's), the gap between the QNC and packet 
forwarding curves increases. However, there is a drawback in 
using BP decoder for some SNR values, especially when the 
sparsity factor of messages, ^, is high (i.e. there is not a high 
correlation between X v 's). Such cases can be explained to be 
a result of propagation of quantization noises, in the network, 
which increases the noise power in the measurements. 

VI. Conclusions 

We have made improvements in the throughput of sensor 
networks, by introducing a network coding based approach for 
transmission of correlated sensed data to a gateway node. Con- 
ventional linear network coding is joined with the concepts of 
Bayesian compressed sensing to efficiently embed distributed 
source coding in network coding. On the other hand, belief 
propagation has helped us to discuss on near optimal decoding 
of quantized network coded messages, while computational 
resource constraints were intended to be met. Our simulation 
results show significant savings for QNC in terms of delivery 
delay, when compared with conventional packet forwarding. 



Moreover, using the proposed BP based MMSE decoder for 
QNC scenario helped us to require a smaller delivery delay, 
for (relatively) low SNR values. As a lacking point in the 
studies of BP based decoding, we are still working to derive 
theoretical bounds on the performance of our decoder. This 
would give us a better understanding about the optimality 
of adopted decoder, with respect to the infinite block length 
information theoretic bounds. 
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